-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
avro schema validation #105
avro schema validation #105
Conversation
switch msg.(type) { | ||
case []byte: | ||
msgBytes = msg.([]byte) | ||
if err := json.Unmarshal(msgBytes, &message); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
myData := []byte({"username": "john", "age": 30}
)
When json unmarshal the above data, int
type is converted to float64
thus causing schema validation error. So when user is defining avro schema, schema can't have int type. It should be double rather than int.
https://github.com/hamba/avro#types-conversions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any workaround to solve this? otherwise we should mention it on the UI as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any workaround to solve this? otherwise we should mention it on the UI as well
I can think of the below solution. But this solution converts int to int64. Still we need to add note in the UI.
So i reckon leave it as it is and add note in the UI.
https://go.dev/play/p/SkG6qlolz1B
myData := `{"username": "John", "age": 30}`
var m map[string]interface{}
dec := json.NewDecoder(strings.NewReader(myData))
dec.UseNumber()
err := dec.Decode(&m)
if err != nil {
log.Fatal(err)
}
m["age"], err = m["age"].(json.Number).Int64()
if err != nil {
log.Fatal(err)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok totally agree
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this thing happening on other SDKs as well or only in Go? if no so let's add a comment about it in the readme file of Go sdk it is enough
switch msg.(type) { | ||
case []byte: | ||
msgBytes = msg.([]byte) | ||
if err := json.Unmarshal(msgBytes, &message); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any workaround to solve this? otherwise we should mention it on the UI as well
return nil, memphisError(err) | ||
} | ||
|
||
default: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In protobuf we allow users to send map[string]interface{} in order people won't have to hole the proto file locally, is it possible here as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I've added it now.
Related to superstreamlabs/memphis#1107